Neural reward representations enable utilitarian welfare maximization.

From deciding which meal to prepare for our guests to trading-off the pro-environmental effects of climate protection measures against their economic costs, we often must consider the consequences of our actions for the well-being of others (welfare). Vexingly, the tastes and views of others can vary widely. To maximize welfare according to the utilitarian philosophical tradition, decision makers facing conflicting preferences of others should choose the option that maximizes the sum of subjective value (utility) of the entire group. This notion requires comparing intensities of preferences across individuals. However, it remains unclear whether such comparisons are possible at all, and (if they are possible) how they might be implemented in the brain. Here, we show that female and male participants can both learn the preferences of others by observing their choices, and represent these preferences on a common scale to make utilitarian welfare decisions. On the neural level, multivariate support vector regressions revealed that a distributed activity pattern in the ventromedial prefrontal cortex (VMPFC), a brain region previously associated with reward processing, represented preference strength of others. Strikingly, also the utilitarian welfare of others was represented in the VMPFC and relied on the same neural code as the estimated preferences of others. Together, our findings reveal that humans can behave as if they maximized utilitarian welfare using a specific utility representation and that the brain enables such choices by repurposing neural machinery processing the reward others receive.Significance statement In many situations politicians and civilians strive to maximize the welfare of social groups. If the preferences of group members are in conflict, identifying the utilitarian welfare-maximizing option requires that decision makers can compare the strengths of conflicting preferences on a common scale. Yet, there is a fundamental lack of understanding which brain mechanisms enable such comparisons of conflicting utilities. Here, we show that brain regions involved in reward processing compute welfare comparisons by representing the preferences of others with a common neural code. This provides a neurobiological mechanism to compute utilitarian welfare maximization as desired by moral philosophy in the Humean tradition.

Adaptive coding of reward in schizophrenia, its change over time and relation to apathy.

Adaptive coding of reward is the process by which neurons adapt their response to the context of available compensations. Higher rewards lead to a stronger brain response, but the increase of the response depends on the range of available rewards. A steeper increase is observed in a narrow range, and a more gradual slope in a wider range. In schizophrenia, adaptive coding appears affected in different domains, and in the reward domain in particular. Here we tested adaptive coding of reward in a large group of patients with schizophrenia (N = 86) and controls (N = 66). We assessed 1) the association between adaptive coding deficits and symptoms; 2) the longitudinal stability of deficits (the same task was performed three months apart); 3) the stability of results between two experimental sites. We used fMRI and the Monetary Incentive Delay task to assess participant' adaptation to two different reward ranges: a narrow and a wide range. We used a region of interest analysis, evaluating adaptation within striatal and visual regions. Patients and controls underwent a full demographic and clinical assessment. We found reduced adaptive coding in patients, due to a decreased slope in the narrow reward range, with respect to that of control participants in striatal but not visual regions. This pattern was observed at both research sites. Upon re-test, patients increased their narrow range slopes, showing improved adaptive coding, whereas controls slightly reduced them. At re-test, patients with overly steep slopes in the narrow range also showed higher levels of negative symptoms. Our data confirm deficits in reward adaptation in schizophrenia and reveal a practice effect in patients, leading to improvement, with steeper slopes upon retest. However, in some patients, an overly steep slope may result in poor discriminability of larger rewards, due to early saturation of the brain response. Together, the loss of precision of reward representation in new (first exposure, underadaptation) and more familiar (re-test, overadaptation) situations may contribute to the multiple motivational symptoms in schizophrenia.

Bayesian reinforcement learning: A basic overview.

We and other animals learn because there is some aspect of the world about which we are uncertain. This uncertainty arises from initial ignorance, and from changes in the world that we do not perfectly know; the uncertainty often becomes evident when our predictions about the world are found to be erroneous. The Rescorla-Wagner learning rule, which specifies one way that prediction errors can occasion learning, has been hugely influential as a characterization of Pavlovian conditioning and, through its equivalence to the delta rule in engineering, in a much wider class of learning problems. Here, we review the embedding of the Rescorla-Wagner rule in a Bayesian context that is precise about the link between uncertainty and learning, and thereby discuss extensions to such suggestions as the Kalman filter, structure learning, and beyond, that collectively encompass a wider range of uncertainties and accommodate a wider assortment of phenomena in conditioning.

Characterizing Human Habits in the Lab.

Habits pose a fundamental puzzle for those aiming to understand human behavior. They pervade our everyday lives and dominate some forms of psychopathology but are extremely hard to elicit in the lab. In this Registered Report, we developed novel experimental paradigms grounded in computational models, which suggest that habit strength should be proportional to the frequency of behavior and, in contrast to previous research, independent of value. Specifically, we manipulated how often participants performed responses in two tasks varying action repetition without, or separately from, variations in value. Moreover, we asked how this frequency-based habitization related to value-based operationalizations of habit and self-reported propensities for habitual behavior in real life. We find that choice frequency during training increases habit strength at test and that this form of habit shows little relation to value-based operationalizations of habit. Our findings empirically ground a novel perspective on the constituents of habits and suggest that habits may arise in the absence of external reinforcement. We further find no evidence for an overlap between different experimental approaches to measuring habits and no associations with self-reported real-life habits. Thus, our findings call for a rigorous reassessment of our understanding and measurement of human habitual behavior in the lab.

The social transmission of empathy relies on observational reinforcement learning.

Theories of moral development propose that empathy is transmitted across individuals. However, the mechanisms through which empathy is socially transmitted remain unclear. Here, we combine computational learning models and functional MRI to investigate whether, and if so, how empathic and non-empathic responses observed in others affect the empathy of female observers. The results of three independent studies showed that watching empathic or non-empathic responses generates a learning signal that respectively increases or decreases empathy ratings of the observer. A fourth study revealed that the learning-related transmission of empathy is stronger when observing human rather than computer demonstrators. Finally, we show that the social transmission of empathy alters empathy-related responses in the anterior insula, i.e., the same region that correlated with empathy baseline ratings, as well as its functional connectivity with the temporoparietal junction. Together, our findings provide a computational and neural mechanism for the social transmission of empathy that accounts for changes in individual empathic responses in empathic and non-empathic social environments.

Acetylcholine and noradrenaline enhance foraging optimality in humans.

Foraging theory prescribes when optimal foragers should leave the current option for more rewarding alternatives. Actual foragers often exploit options longer than prescribed by the theory, but it is unclear how this foraging suboptimality arises. We investigated whether the upregulation of cholinergic, noradrenergic, and dopaminergic systems increases foraging optimality. In a double-blind, between-subject design, participants (N = 160) received placebo, the nicotinic acetylcholine receptor agonist nicotine, a noradrenaline reuptake inhibitor reboxetine, or a preferential dopamine reuptake inhibitor methylphenidate, and played the role of a farmer who collected milk from patches with different yield. Across all groups, participants on average overharvested. While methylphenidate had no effects on this bias, nicotine, and to some extent also reboxetine, significantly reduced deviation from foraging optimality, which resulted in better performance compared to placebo. Concurring with amplified goal-directedness and excluding heuristic explanations, nicotine independently also improved trial initiation and time perception. Our findings elucidate the neurochemical basis of behavioral flexibility and decision optimality and open unique perspectives on psychiatric disorders affecting these functions.

Testosterone administration modulates inequality aversion in healthy males: evidence from computational modeling.

Fairness concerns play a prominent role in promoting cooperation in human societies. Social preferences involving fairness concern have been associated with individual testosterone levels. However, the causal effects of testosterone administration on fairness-related decision making remain to be elucidated. Here, we used a randomized, double-blind, between-participant design and administered testosterone or placebo gel to 120 healthy young men. Three hours after administration, participants performed a modified Dictator Game from behavioral economics, in which they were asked to choose one of two monetary allocations between themselves and anonymous partners. Participants were either in a position of advantageous inequality (i.e., endowed with more than others) or disadvantageous inequality (i.e., endowed with less than others). Computational modeling showed that inequality-related preferences explained behavior better than competing models. Importantly, compared with the placebo group, the testosterone group showed significantly reduced aversion to advantageous inequality but enhanced aversion to disadvantageous inequality. These findings suggest that testosterone facilitates decisions that prioritize selfish economic motives over fairness concerns, which in turn may boost status-enhancing behaviors.

Toward a Unifying Account of Dopamine's Role in Cost-Benefit Decision Making.

Dopamine is thought to play a crucial role in cost-benefit decision making, but so far there is no consensus on the precise role of dopamine in decision making. Here, we review the literature on dopaminergic manipulations of cost-benefit decision making in humans and evaluate how well different theoretical accounts explain the existing body of evidence. Reduced D2 stimulation tends to increase the willingness to bear delay and risk costs (i.e., wait for later rewards, take riskier options), while increased D1 and D2 receptor stimulation increases willingness to bear effort costs. We argue that the empirical findings can best be explained by combining the strengths of two theoretical accounts: in cost-benefit decision making, dopamine may play a dual role both in promoting the pursuit of psychologically close options (e.g., sooner and safer rewards) and in computing which costs are acceptable for a reward at stake. Moreover, we identify several limiting factors in the study designs of previous investigations that prevented a fuller understanding of dopamine's role in value-based choice. Together, the proposed theoretical framework and the methodological suggestions for future studies may bring us closer to a unifying account of dopamine in healthy and impaired cost-benefit decision making.

Sensory perception relies on fitness-maximizing codes.

Sensory information encoded by humans and other organisms is generally presumed to be as accurate as their biological limitations allow. However, perhaps counterintuitively, accurate sensory representations may not necessarily maximize the organism's chances of survival. To test this hypothesis, we developed a unified normative framework for fitness-maximizing encoding by combining theoretical insights from neuroscience, computer science, and economics. Behavioural experiments in humans revealed that sensory encoding strategies are flexibly adapted to promote fitness maximization, a result confirmed by deep neural networks with information capacity constraints trained to solve the same task as humans. Moreover, human functional MRI data revealed that novel behavioural goals that rely on object perception induce efficient stimulus representations in early sensory structures. These results suggest that fitness-maximizing rules imposed by the environment are applied at early stages of sensory processing in humans and machines.

A process model account of the role of dopamine in intertemporal choice.

Theoretical accounts disagree on the role of dopamine in intertemporal choice and assume that dopamine either promotes delay of gratification by increasing the preference for larger rewards or that dopamine reduces patience by enhancing the sensitivity to waiting costs. Here, we reconcile these conflicting accounts by providing empirical support for a novel process model according to which dopamine contributes to two dissociable components of the decision process, evidence accumulation and starting bias. We re-analyzed a previously published data set where intertemporal decisions were made either under the D2 antagonist amisulpride or under placebo by fitting a hierarchical drift diffusion model that distinguishes between dopaminergic effects on the speed of evidence accumulation and the starting point of the accumulation process. Blocking dopaminergic neurotransmission not only strengthened the sensitivity to whether a reward is perceived as worth the delay costs during evidence accumulation (drift rate) but also attenuated the impact of waiting costs on the starting point of the evidence accumulation process (bias). In contrast, re-analyzing data from a D1 agonist study provided no evidence for a causal involvement of D1R activation in intertemporal choices. Taken together, our findings support a novel, process-based account of the role of dopamine for cost-benefit decision making, highlight the potential benefits of process-informed analyses, and advance our understanding of dopaminergic contributions to decision making.

Association of pessimism with cardiovascular events and all-cause mortality.

Poor psychological health is associated with Takotsubo cardiomyopathy, cardiac syndrome X, coronary microcirculatory dysfunction, peripheral artery disease, or spontaneous coronary artery dissection. Data regarding pessimism, cardiovascular disease (CVD) events and mortality and all-cause mortality remained inconclusive. This systematic review and meta-analysis aim to provide an overview of the association between pessimism, CVD outcomes and mortality. A systematic search of electronic databases was conducted from inception through July 2022 for studies evaluating pessimism and adverse outcomes. A total of 17 studies published between 1966 and July 2022 met our inclusion criteria, for a total of 232,533 individuals. Pooled hazard ratios were calculated in random-effects meta-analyses. Based on pooled analysis of adjusted HRs, pessimism was associated with adjusted HR of 1.13 (95% CI 1.07-1.19) for all-cause mortality with minimal heterogeneity (I2 = 28.5%). Based on pooled analysis of adjusted HRs, pessimism was associated with adjusted HR of 1.30 (95% CI 0.43-3.95) for CHD mortality, adjusted HR of 1.41 (95% CI 1.05-1.91) for CVD mortality, and adjusted HR of 1.43 (95% CI 0.64-3.16) for stroke. In conclusion, pessimism seems to be significantly associated with a higher risk for and poorer outcomes from CVD events than optimistic styles. There are genetic and other bases for these life approaches, but behavioral, cognitive and meditative interventions can modify patients' level of pessimism, hopefully leading to better medical outcomes. Testing this theory would yield highly useful and practical data for clinical care.

Comparing adaptive coding of reward in bipolar I disorder and schizophrenia.

Deficits in neural processing of reward have been described in both bipolar disorder (BD) and schizophrenia (SZ), but it remains unclear to what extent these deficits are caused by similar mechanisms. Efficient reward processing relies on adaptive coding which allows representing large input spans by limited neuronal encoding ranges. Deficits in adaptive coding of reward have previously been observed across the SZ spectrum and correlated with total symptom severity. In the present work, we sought to establish whether adaptive coding is similarly affected in patients with BD. Twenty-five patients with BD, 27 patients with SZ and 25 healthy controls performed a variant of the Monetary Incentive Delay task during functional magnetic resonance imaging in two reward range conditions. Adaptive coding was impaired in the posterior part of the right caudate in BD and SZ (trend level). In contrast, BD did not show impaired adaptive coding in the anterior caudate and right precentral gyrus/insula, where SZ showed deficits compared to healthy controls. BD patients show adaptive coding deficits that are similar to those observed in SZ in the right posterior caudate. Adaptive coding in BD appeared more preserved as compared to SZ participants especially in the more anterior part of the right caudate and to a lesser extent also in the right precentral gyrus. Thus, dysfunctional adaptive coding could constitute a fundamental deficit in severe mental illnesses that extends beyond the SZ spectrum.

Increased Meal Size but Reduced Meal-Stimulated Plasma Cholecystokinin Concentrations in Women With Obesity.

To better understand the physiological basis of obesity in women, we investigated whether obesity or menstrual cycle phase affects laboratory test-meal size or meal-stimulated plasma cholecystokinin (CCK) concentration. Women with healthy weight (body mass index [BMI] of 18.5-24.9 kg/m2, N = 16) or obesity (BMI 30-39.9 kg/m2, N = 20) were tested once in the late-follicular or peri-ovulatory phase (LF/PO) and once in the mid-luteal phase (ML). Meals of ham sandwiches were offered and blood was sampled. Menstrual cycle phases were verified with participants' reports of menses and measurements of progesterone and luteinizing hormone (LH) concentrations. Women with obesity ate significantly larger meals than women with healthy weight, (mean, 711 [95% CI, 402-1013] kJ, P = 0.001, during the LF/PO and 426 [105-734] kJ, P = 0.027, larger during the ML). Women with healthy weight ate smaller meals during LF/PO than ML (decrease, 510 [192-821 kJ], P = 0.008), but women with obesity did not (decrease, 226 [-87-542] kJ, P = 0.15). CCK concentrations 18 to 30 minutes after meal onset were lower in women with obesity than in women with healthy weight during LF/PO (3.6 [3.1-4.1] vs 6.1 [4.5-7.7] pmol/L; P = 0.004), but not during ML, with a significant interaction effect (1.8 [1.2-2.4] pmol/L, P = 0.048). Women with obesity consumed larger meals than women with healthy weight but displayed reduced meal-stimulated plasma CCK concentrations. These data are consistent with the hypothesis that a defect in CCK secretion compromises satiation in obese women and contributes to the development or maintenance of obesity.

Adaptive coding occurs in object categorization and may not be associated with schizotypal personality traits.

Processing more likely inputs with higher sensitivity (adaptive coding) enables the brain to represent the large range of inputs coming in from the world. Healthy individuals high in schizotypy show reduced adaptive coding in the reward domain but it is an open question whether these deficits extend to non-motivational domains, such as object categorization. Here, we develop a novel variant of a classic task to test range adaptation for face/house categorization in healthy participants on the psychosis spectrum. In each trial of this task, participants decide whether a presented image is a face or a house. Images vary on a face-house continuum and appear in both wide and narrow range blocks. The wide range block includes most of the face-house continuum (2.50-97.5% face), while the narrow range blocks limit inputs to a smaller section of the continuum (27.5-72.5% face). Adaptive coding corresponds to better performance for the overlapping smaller section of the continuum in the narrow range than in the wide range block. We find that participants show efficient use of the range in this task, with more accurate responses in the overlapping section for the narrow range blocks relative to the wide range blocks. However, we find little evidence that range adaptation in our object categorization task is reduced in healthy individuals scoring high on schizotypy. Thus, reduced range adaptation may not be a domain-general feature of schizotypy.

Exogeneous testosterone increases sexual impulsivity in heterosexual men.

Testosterone has been hypothesized to promote sexual motivation and behavior. However, experimental evidence in healthy humans is sparse and rarely establishes causality. The present study investigated how testosterone affects delay of gratification for sexual rewards. We administered a single dose of testosterone to healthy young males in a double-blind, placebo-controlled, between-participant design (N = 140). Participants underwent a sexual delay discounting task, in which they made a choice between a variable larger-later option (i.e., waiting longer to view a sexual picture for a longer duration) and a smaller-sooner option (i.e., waiting for a fixed shorter period of time to view the same picture for a shorter duration). We found that testosterone administration increased preference for the smaller-sooner option and induced steeper discounting for the delayed option. These findings provide direct experimental evidence that rapid testosterone elevations increase impulsivity for sexual rewards and represent an important step towards a better understanding of the neuroendocrine basis of sexual motivation in humans.

Analysis of individual differences in neurofeedback training illuminates successful self-regulation of the dopaminergic midbrain.

The dopaminergic midbrain is associated with reinforcement learning, motivation and decision-making - functions often disturbed in neuropsychiatric disorders. Previous research has shown that dopaminergic midbrain activity can be endogenously modulated via neurofeedback. However, the robustness of endogenous modulation, a requirement for clinical translation, is unclear. Here, we examine whether the activation of particular brain regions associates with successful regulation transfer when feedback is no longer available. Moreover, to elucidate mechanisms underlying effective self-regulation, we study the relation of successful transfer with learning (temporal difference coding) outside the midbrain during neurofeedback training and with individual reward sensitivity in a monetary incentive delay (MID) task. Fifty-nine participants underwent neurofeedback training either in standard (Study 1 N = 15, Study 2 N = 28) or control feedback group (Study 1, N = 16). We find that successful self-regulation is associated with prefrontal reward sensitivity in the MID task (N = 25), with a decreasing relation between prefrontal activity and midbrain learning signals during neurofeedback training and with increased activity within cognitive control areas during transfer. The association between midbrain self-regulation and prefrontal temporal difference and reward sensitivity suggests that reinforcement learning contributes to successful self-regulation. Our findings provide insights in the control of midbrain activity and may facilitate individually tailoring neurofeedback training.

Learning from Ingroup Experiences Changes Intergroup Impressions.

Humans form impressions toward individuals of their own social groups (ingroup members) and of different social groups (outgroup members). Outgroup-focused theories predict that intergroup impressions are mainly shaped by experiences with outgroup individuals, while ingroup-focused theories predict that ingroup experiences play a dominant role. Here we test predictions from these two psychological theories by estimating how intergroup impressions are dynamically shaped when people learn from both ingroup and outgroup experiences. While undergoing fMRI, male participants had identical experiences with different ingroup or outgroup members and rated their social closeness and impressions toward the ingroup and the outgroup. Behavioral results showed an initial ingroup bias in impression ratings which was significantly reduced over the course of learning, with larger effects in individuals with stronger ingroup identification. Computational learning models revealed that these changes in intergroup impressions were predicted by the weight given to ingroup prediction errors. Neurally, the individual weight for ingroup prediction errors was related to the coupling between the left inferior parietal lobule and the left anterior insula, which, in turn, predicted learning-related changes in intergroup impressions. Our findings provide computational and neural evidence for ingroup-focused theories, highlighting the importance of ingroup experiences in shaping social impressions in intergroup settings.Significance Statement:Living in multicultural societies, humans interact with individuals of their own social groups (ingroup members) and of different social groups (outgroup members). However, little is known about how people learn from the mixture of ingroup and outgroup interactions, the most natural experiences in current societies. Here, participants had identical, intermixed experiences with different ingroup and outgroup individuals and rated their closeness and impressions toward the ingroup and the outgroup. Combining computational models and fMRI, we find that the weight given to ingroup experiences (ingroup prediction errors) is the main source of intergroup impression change, captured by changes in connectivity between the parietal lobe and insula. These findings highlight the importance of ingroup experiences in shaping intergroup impressions in complex social environments.

Cerebellar and cortico-striatal-midbrain contributions to reward-cognition processes and apathy within the psychosis continuum.

Negative symptoms in the psychosis continuum are linked to impairments in reward processing and cognitive function. Processes at the interface of reward processing and cognition and their relation to negative symptoms remain little studied, despite evidence suggestive of integration in mechanisms and neural circuitry. Here, we investigated brain activation during reward-dependent modulation of working memory (WM) and their relationship to negative symptoms in subclinical and early stages of the psychosis continuum. We included 27 persons with high schizotypal personality traits and 23 patients with first episode psychosis as well as 27 healthy controls. Participants underwent functional magnetic resonance imaging while performing an established 2-back WM task with two reward levels (5 CHF vs. no reward), which allowed us to assess common reward-cognition regions through whole-brain conjunction analyses and to investigate relations with clinical scores of negative symptoms. As expected for behavior, reward facilitated performance while cognitive load diminished it. At the neural level, the conjunction of high reward and high cognitive load contrasts across the psychosis continuum showed increased hemodynamic activity in the thalamus and the cerebellar vermis. During high cognitive load, more severe apathy but not diminished expression in the psychosis continuum was associated with reduced activity in right lateral orbitofrontal cortex, midbrain, posterior vermal cerebellum, caudate and lateral parietal cortex. Our results suggest that hypoactivity in the cerebellar vermis and the cortical-striatal-midbrain-circuitry in the psychosis continuum relates to apathy possibly via impaired flexible cognitive resource allocation for effective goal pursuit.

Is It Worth It? Obesity Affects Snack Food Valuation Across the Menstrual Cycle.

BACKGROUND: The importance of menstrual cycle physiology in appetite and obesity is poorly understood. We investigated the effects of body mass index (BMI), menstrual cycle phase and sweet and salty taste on monetary valuation of snack foods. METHODS: We recruited 72 women and after the application of in- and exclusion criteria 31 participants with healthy weight and 25 with obesity remained. The participants completed a willingness to pay (WTP) task to measure subjective value of 30 snack food items in the pre-ovulatory and mid-luteal cycle phases. RESULTS: Generalized linear mixed model (GLMM) analysis revealed that BMI, cycle phase and snack taste interacted to influence WTP (-0.15 [-0.22, -0.03], p = 0.002). Hence, WTP was inversely related to BMI, but the strength of the relation depended on cycle phase and taste. The WTP of participants with healthy weight for salty taste changed across cycle phase but the WTP for sweet taste was not affected by cycle phase. Moreover, the cycle effect for the salty snacks ceased in participants with obesity. CONCLUSION: The inverse effect of BMI on WTP valuation of snack foods contrasts with the positive effect of BMI on pleasantness ratings for milkshakes by the same women that we previously reported. This indicates that the two measures reflect different aspects of food-related valuative processing in obesity. Furthermore, the WTP data suggest that the selection of salty snacks may differ from that of sweet snacks in the pre-ovulatory phase of the menstrual cycle for individuals of healthy weight. The cycle phase does not seem to affect food valuation of participants with obesity. These findings are relevant to understanding and treating obesity in women.